Alternative models NEW

In order to discern which model predictions are specific to a rational meta-memory model, we show predictions of two alternative models. The Random model randomly samples fixation durations from the empirical distribution. We also A more sophisticated Random Commitment model was developed to account for the empirical fact that last fixations are considerably longer than all other fixations. This model samples the total number of fixations on each trial from the empirical distribution and then samples the duration of each from the empirical distribution of non-final fixations. If a target is not recalled before reaching the final fixation, the model continues to fixate on the current cue until the target is recalled or the 15 second time limit is reached.

Measuring memory strength NEW

The model predicts that people will spend more time looking at the cue for which the memory of the corresponding target is stronger, that is the cue for which \(p\) is higher. Unfortunately, we cannot measure \(p\). However, we can collect a noisy signal of this parameter using an auxiliary task. Concretely, we use reaction time in a 2-AFC task in which participants are presented with a word and must select the matching image.

To map this measure onto the model’s \(p\) parameter, we take advantage of the fact that the expected time to reach threshold in the model is \(E[t] = \theta / p\) which implies that \(\log E[t] = \log(\theta) - \log(p)\). This suggests that, in broad strokes, the \(p\) parameter should be log-linearly related to response times. The exact nature of this relationship is unclear, however, given that the 2-AFC task is quite different from a cued recall task. Thus, we simply normalize (Z-score) the \(\log(p)\) parameter and the 2AFC-RT measure (the latter within subject) to put them on roughtly the same scale. Finally, to account for the fact that the 2AFC measure is very noisy, we corrupt \(\log(p)\) with \(\sigma=3\) Gaussian noise.

In the simulations, we sample the parameter \(p\) from a \(\text{Beta}(2, 6)\) distribution, which was chosen because it makes some of the plots look better. This corresponds to an assumption about the range of accumulation rates (memory strengths) that are likely.

Timeline NEW

• Exposure: each of 40 pairs shown once
• Test: cued-recall image to word, no feedback, each image shown once
• Exposure
• Distractor: 30 seconds of simple arithmetic
• Test: each image shown twice. Performance used as memory strength index
• Critical: double-cue recall, each pair shown once (two per trial, so twenty trials)

Fixation time course

The key prediction of a dynamic optimal sampling model such as the one we propose is that a cue with stronger memory strength should receive greater attention. This is because such a cue can be rememberd more quickly and thus presents a lower-cost route to recall. Furthermore, this focusing effect should be stronger later in the trial, when the agent has had sufficient time to determine which cue is stronger. This is illustrated in the following plot, which shows the probability of fixating the stronger cue as a function of progress in the trial (0 is stimuli onset, 1 is response onset) and the size of the difference in strength.

normalized_timestep = function(long) {
    long %>% 
        group_by(trial) %>%
        mutate(prop_duration = duration / sum(duration)) %>% 
        ungroup() %>% 
        mutate(n_step=round(prop_duration * 100)) %>% 
        uncount(n_step) %>% 
        group_by(trial) %>% 
        mutate(normalized_timestep = row_number())
}

long %>% 
    normalized_timestep %>% 
    drop_na(strength_diff) %>% 
    ggplot(aes(normalized_timestep/100, fix_stronger, group = strength_diff, color=strength_diff)) +
    geom_smooth(se=F) + 
    ylim(0, 1) +
    facet_grid(~name) +
    labs(x="Normalized Time", y="Probability Fixate\nStronger Cue", color="Strength Difference") +
    geom_hline(yintercept=0.5) +
    theme(legend.position="top")

In both the optimal model simulations and the human data, the probability of fixating the stronger cue steadily increases over the course of the trial, and this increase is more pronounced when there is a larger difference in strength.

In the random model, we only see a slight spike at the end of the trial. This spike is due to the fact that the model always remembers the item it looks at last, and it is also more likely to remember the item with a stronger memory.

The random commitment model does not show this effect because the item it fixates last (and thus remembers) is usually determined by the random commitment.

Overall fixation proportion

Consisent with a trend to look more at stronger-memory items, we see that the total proportion of time spent fixating on an item increases with its memory strength, relative to the other item.

df %>% 
    filter(n_pres >= 2) %>% 
    regress(rel_strength, prop_first)

Last-fixation effect

However, the overal proportion effect is qualified by an interaction with the target of the last fixation. People overwhelmingly tend to select the last-viewed cue. In value-based decision making, it has been suggested that this “last-fixation effect” could explain away what looks to be evidence for adaptive attention allocation. Briefly, the last fixation is correlated with both fixation proportion and relative strength, and this could be entirely driving the correlation between strength and fixation proportion.

df %>% 
    filter(n_pres >= 2) %>% 
    regress_interaction(rel_strength, last_pres, prop_first)

It is worth noting, however, that we cannot reproduce an effect of comparable strength in the random model (with some effort, but not quite an exhaustive search, which we will do at some point).

Why is it that the optimal model and human data both show the total proportion effect but the random model seemingly cannot? We think it is due to a combination of the following two effects:

Last fixations are longer

long %>% 
    # filter(name == "Human") %>% 
    ggplot(aes(last_fix==1, duration)) + 
    stat_summary(fun.data=mean_cl_boot, geom="bar", fill="white", color="black") +
    stat_summary(fun.data=mean_cl_boot, geom="errorbar", width=0.2) +
    facet_grid(~name) + 
    scale_x_discrete(name="Fixation Type", labels=c("Non-final", "Final")) +
    ylab("Duration")

last_diff = long %>% 
    filter(name == "Human") %>% 
    with(tapply(duration, last_fix, mean)) %>% 
    diff

long %>% 
    filter(name == "Human") %>% 
    mutate(is_last = int(last_fix==1)) %>% 
    lmer(duration ~ is_last + (is_last|wid), data=.) %>% 
    summ

The reason we see this in the optimal model is that the trial doesn’t stop when the model decides which cue has a stronger memory (as in value-based and perceptual tasks). Instead, the model needs to continue fixating the that cue until it remembers it. By this logic, the long last fixation is evidence that at some point people commit to remembering one of the cues. This is what initially motivated the random commitment model.

Side note: this is a cool result because we see the opposite (shorter final fixations) in value-based and perceptual decisions. The shorter last fixation effects is typically explained as the result of crossing a threshold and cutting off the final fixation.

Strength predicts last fixation

df %>% 
    mutate(last_pres_first = as.numeric(last_pres == "first")) %>% 
    regress(rel_strength, last_pres_first)

This effect occurs mechanistically (i.e., in the random model) because (1) the currently fixated cue is almost always the one remembered, (2) remembering the cue terminates the trial, making the current fixation the final one, and (3) the stronger cue is more likely to be remembered.

However, this effect doesn’t occur in the random commitment model because the cue it fixates last is usually determined by the random commitment, not by crossing the threshold (only the latter being more likely for stronger cues). Comparing these two effects, we see that the normal random model captures the second but not the first, while the random comittment model captures the first but not the second. Only the optimal model and humans show both effects.

Individual fixation durations

Although the random models we designed did not demonstrate the smoothly increasing probability of fixating the stronger cue, nor the overal proportion effect, it is possible that some other confound related to the last fixation explain the effects. For this reason, it is informative to look at individual non-final fixation durations.

Because we are excluding final fixations, different participants are better represented in some part of the x axis than others. This produces Simpon’s-paradox-like effects. Furthermore, because many participants have few trials with more than two or three fixations, it is difficult to account for this with random effects models (few data points per individual). For this reason, we plot and analyze fixation durations that have first been normalized (z-scored) by the mean and SD duration for that participant across all non-final fixations (there are not substantial differences in duration by fixation number). We still employ mixed effects models to account for individual differences in sensitivity to cue strength.

df %>% 
    filter(n_pres >= 2) %>% 
    regress(strength_first, first_pres_time, mixed=MIXED_DURATIONS)

df %>% 
    filter(n_pres >= 3) %>% 
    regress(rel_strength, second_pres_time, mixed=MIXED_DURATIONS)

df %>% 
    filter(n_pres >= 4) %>% 
    regress(rel_strength, third_pres_time, mixed=MIXED_DURATIONS)

Probability of remembering first word

df %>%
    filter(response_type == "correct") %>% 
    mutate(choose_first = int(choose_first)) %>% 
    regress(rel_strength, choose_first) +
    ylab("Prob Select First Cue") + ylim(0, 1)

Reaction time by chosen strength

df %>% 
    filter(response_type == "correct") %>% 
    regress(chosen_strength, rt)

Last fixation duration by strength

df %>% 
    filter(response_type == "correct") %>% 
    filter(n_pres > 0) %>% 
    mutate(
        last_pres_time = map_dbl(presentation_times, last),
        last_rel_strength = if_else(last_pres == "first", rel_strength, 1 - rel_strength),
        last_strength = if_else(last_pres == "first", strength_first, strength_second)
    ) %>% 
    regress(last_strength, last_pres_time)

Response types

Note: The model can only predict correct answers and timeouts.

raw_df %>% 
    with((proportions(table(name, response_type), margin=1))) %>% 
    kable(digits=2)

Reaction time

df %>% 
    ggplot(aes(rt)) +
    geom_density() +
    facet_grid(~name) +
    labs(x="Reaction Time", y="Density")

Number of fixations

df %>% 
    filter(n_pres < 10) %>% 
    ggplot(aes(n_pres, ..prop..)) +
    geom_bar() +
    facet_grid(~name) +
    labs(x="Number of Fixations", y="Proportion of Trials")